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ABSTRACT 


This  paper  applies  the  theory  of  Semi-Markov  Processes  to  the  construction 
of  a  stochastic  model  for  interpreting  data  obtained  from  clinical  trials.  The 
model  characterizes  the  patient  as  being  in  one  of  a  finite  number  of  states  at 
any  given  time  with  an  arbitrary  probability  distribution  to  describe  the  length 
of  stay  in  a  state.  Transitions  between  states  are  assumed  to  be  chosen 
according  to  a  stationary  finite  Markov  chain. 

Other  attempts  have  been  made  to  develop  stochastic  models  of  clinical 
trials.  However,  these  have  all  been  essentially  Markovian  with  constant 
transition  probabilities  which  implies  that  the  distribution  of  time  spent  during 
a  visit  to  a  state  is  exponential  (or  geometric  for  discrete  Markov  chains). 

Markov  models  need  also  to  assume  that  the  transitions  in  the  state  of  a  patient 
depend  only  on  absolute  time  whereas  the  Semi-Markov  model  assumes  that 
transitions  depend  on  time  relative  to  a  patient.  Thus  the  models  are  applicable 
to  degenerative  diseases  (cancer,  acute  leukemia),  while  Markov  models  with 
time  dependent  transition  probabilities  are  applicable  to  colds  and  epidemic 
diseases.  In  this  paper  the  Laplace  transforms  are  obtained  for  (i)  probability 
of  being  in  a  state  at  time  t,  (ii)  probability  distribution  to  reach  absorption 
state  and  (iii)  the  probability  distribution  of  the  first  passage  times  to  go  from 
initial  states  to  transient  or  absorbing  states,  transient  to  transient,  and  transieni 
to  absorbing.  The  model  is  applied  to  a  clinical  study  of  Acute  Leukemia  which 
have  been  treated  with  methotrexate  and  6-mercaptopurine.  The  agreement  of 
the  data  and  the  model  is  very  good. 
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1.  Introduction 

There  are  many  diseases  which  can  be  characterized  by  the  patient  being 
in  one  of  a  finite  number  of  states  such  that  the  sojourn  time  within  a  state  is 
a  random  variable.  For  example,  many  clinical  trials  can  be  described  as  a 
course  of  treatment  given  to  a  patient  who  is  in  a  state  of  illness  or  relapse, 
in  the  hope  that  the  therapy  will  ultimately  cure  the  patient  or  bring  him  into 
some  degree  of  remission.  During  the  course  of  the  treatment  the  patient  may 
be  in  remission  and  relapse  states  several  times.  When  evaluating  such  courses 
of  therapy  in  clinical  trials  it  is  important  to  be  able  to  discuss  various 
statistical  characteristics  of  the  therapy;  i.  e. ,  proportion  cured  or  reaching  a 
remission  state,  time  to  reach  a  given  state  for  the  first  time,  the  first  passage 
time  to  cure  or  death,  the  probability  of  being  cured  or  reaching  remission  as  a 
function  of  the  duration  of  therapy,  etc. 

As  a  possibly  more  concrete  motivation  for  the  theory  which  follows,  we 
mention  the  problem  which  stimulated  our  work.  At  the  present  time  acute 
leukemia  cannot  be  cured  by  any  known  therapy.  It  is  known,  however,  that 
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certain  drugs  can  prolong  a  patient's  life,  or  the  period  of  relative  freedom 
from  symptoms  characteristic  of  the  disease.  Currently  a  large-scale  clinical 
program  is  underway,  sponsored  by  the  National  Institutes  of  Health,  to  test 
the  efficacy  of  many  chemical  agents  on  the  course  of  the  disease,  with  the 
obiect  of  being  able  to  prolong  life,  if  indeed  it  is  not  possible  to  discover 
an  absolute  cure.  Hence,  it  is  necessary  to  have  idealized  models  which  can 
be  used  as  a  framework  for  the  analysis  of  the  data.  Perhaps  the  most  important 
use  of  a  model  is  as  a  guide  in  deciding  the  types  of  data  to  be  taken  during  the 
course  of  a  series  of  clinical  trials.  It  is  the  purpose  of  this  paper  to  discuss 
a  general  stochastic  model  for  the  description  of  the  course  of  chronic  diseases 
and  the  outcome  of  clinical  trials.  Our  model  is  based  on  the  theory  of  semi- 
Markov  processes,  (c.f.  Pyke  [1],  [2]}. 

2.  Earlier  Models 

Several  studies  have  appeared  in  the  literature  with  the  same  object  as  the 
present  paper.  However  these  earlier  works  treat  the  model  as  a  strictly  Markov 
process.  The  first  of  these,  by  Fix  and  Neyman,  [3],  discusses  a  four  state 

continuous  Markov  process  for  the  treatment  of  data  on  cancer  patients.  The  four 
states  were: 

1.  Under  treatment 

2.  Dead  following  treatment  or  operative  death 

3.  Alive,  not  under  treatment  for  cancer,  and 
remaining  under  observation. 
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4.  Lost  after  apparent  recovery,  either  through 
death  not  due  to  cancer  or  through  difficulties 
of  tracing  the  patient. 

The  equations  characterizing  Fix  and  Neyman's  model  are 


dP 

dt 


=  -  AP 


(2.1) 


where  P  is  the  vector  of  state  probabilities,  and  A  is  a  constant  matrix. 

A  formal  solution  to  (2.1)  can  always  be  written  P(t)  =  exp  (-At)  P(0),  hence 
the  probability  of  being  in  any  one  state  at  time  t  can  always  be  written  as  a 
weighted  sum  of  negative  exponential  terms.  Fix  and  Neyman  also  discuss 
estimation  procedures  for  fitting  the  parameters  in  their  model. 

Marshall  and  Goldhamer,  [4],  presented  several  discrete  Markov  chain 
models  as  a  framework  for  the  handling  of  data  in  connection  with  the 
epidemiology  of  mental  disease.  Their  object  was  to  characterize  the  age 
distributions  of  the  mentally  ill  population.  The  states  which  figured  in  their 
analysis  were  the  following: 

1.  Alive,  sane 

2.  Alive,  insane  (mild)  unhospitalized 

3.  Alive,  insane  (severe)  unhospitalized 

4.  Insane,  hospitalized 

5.  Dead,  outside  of  mental  institution  . 

The  authors  give  a  long  discussion  of  their  data  and  estimation  procedures.  Since 
their  model  supposes  as  the  underlying  descriptive  equations 
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P(n  +  1}  =  A  P(n)  (2.2) 

where  P(n)  is  the  vector  of  state  probabilities  after  n  epochs ,  the  form 
of  the  solution  is  always  fixed  to  be  P(n)  =  An  P(0)  . 

The  most  recent  discussion  of  the  Markov  model  has  been  given  by  Ailing, 

[5],  as  an  aid  in  interpreting  data  on  pulmonary  tuberculosis.  Ailing's  model 
uses  the  discrete  Markov  chain,  as  might  be  natural  in  a  situation  involving 
periodic  examinations.  The  model  provided  excellent  agreement  with  experimental 
data. 

All  of  the  Markov  models  (with  constant  transition  probabilities)  have  in 
them  the  assumption  that  the  probability  density  for  a  stay  in  a  given  state  is 
of  the  form  cp(t)  =  X  exp  (-\t)  or  the  discrete  analogue.  In  order  to  introduce 
a  more  general  model  to  describe  the  same  phenomena,  a  somewhat  deeper 
discussion  is  required  of  the  way  in  which  time  is  introduced  as  a  descriptive 

-S 

measure  of  the  disease.  Toward  this  end  we  note  that  time  can  be  involved  in 
the  history  of  a  disease,  either  as  external  or  internal  time  relative  to  the 
patient.  A  strictly  Markov  model  assumes  that  transitions  in  the  state  of  a 
patient  depend  only  on  the  time  external  to  the  patient.  As  an  example,  the 
susceptibility  of  a  person  to  catching  a  cold  may  depend  more  on  the  season  than 
when  he  has  last  had  a  cold.  On  the  other  hand,  the  change  of  state  of  a  person 
with  a  cancerous  disease  will  depend  more  critically  on  how  long  he  has  had 
the  growth  than  on  other  external  circumstances.  It  is  for  the  description  of 
diseases  which  require  that  internal  time  be  taken  into  account  that  the  theory 
of  semi-Markov  processes  is  appropriate. 
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3.  Elements  of  the  Model 

We  shall  consider  a  model  consisting  of  two  mutually  exclusive  sets  of 
states  A  and  T  in  which  the  patients  may  find  themselves.  The  state  A 
corresponds  to  the  set  of  absorbing  states  and  T  refers  to  the  transient  states. 
The  set  A  can  be  void  as  in  the  description  (to  a  good  approximation)  of  the 
cold;  it  may  consist  of  a  single  state,  as  for  example  death  in  the  description 
of  acute  leukemia;  or  it  may  consist  of  two  states,  cure  or  death,  as  in  a 
description  of  tuberculosis.  We  will  assume  stationarity  of  the  processes,  in 
the  sense  that  aging  effects  will  not  be  taken  into  account.  We  further  assume 
the  following: 

(a)  The  duration  of  stay  in  any  state  is  a  random  variable. 

Specifically,  the  probability  density  for  a  stay  in  state  i  which 

3 

is  followed  by  a  stay  in  state  j  will  be  denoted  by  <p ^  (t) 

where  the  <p ^  (t)  may  contain  delta  function  components. 

We  also  define  the  survival  function  (tail  probability  integral) 

00 

=  ^ij(xJdx  •  t3*1) 


(b)  The  probability  of  a  change  from  state  i  to  state  j  in  a 


single  step  (conditional  on  leaving  i  )  is 


Pii 


where  the 


P 


ij 


are  independent  of  time. 


(c)  The  p. d.f.  (p^t)  is  assumed  to  possess  moments  of  any 

order.  The  kth  moment  of  <p^(t)  will  be  denoted  by  m..(k)  . 

3  It  is  particularly  convenient  to  have  two  subscripts  on  these  probability  distribu- 
butions  since  we  may  be  concerned  with  the  termination  of  stay  in  a  state  by 
different  causes.  In  the  present  work,  in  which  we  are  concerned  mainly  with 
an  absorbing  set  of  states,  it  will  be  found  that  only  unconditional  probability 
densities  are  really  necessary. 
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In  addition  to  the  notation  and  assumptions  introduced  above  we  define 

lh(t)  =  Pr  {patient  in  state  i  at  time  t  } 

(*^(t}dt  =  Pr  {patient  leaves  state  i  during  (t,  t  +  dt)  } 

00  / 


^(t)  =  Z  Pij  , 


<Pt{  t)  dt 


(3.2) 


00 

d&iCt)  =  /  ^(x)  dx  . 


Since  we  also  allow  for  the  possibility  that  an  initial  occurrence  of  state 
i  might  have  a  different  p.  d.f. ,  define  t)  as  the  p.  d.f.  of  the  initial 

stay  in  state  i  followed  by  a  stay  in  state  j  .  Also,  m^(k)  wi^  denote  the 
corresponding  kth  moment. 

Matrices  or  vectors  of  quantities  will  be  denoted  by  bold  face  type;  the 
same  symbol  being  used  for  the  matrix  as  are  used  for  its  components. 
Consequently  we  have 


U(t)  =  (U2ct),  u2(t),  ...  un(t) ) 

«(t)  =  (“jCt),  «2(t),  ...  «n(t) ) 

Ht)  <P° {t)  =  (*°(t)  5..) 

M(k)  =  (m.(k)  6..),  M°(k)  =  (m^(k)  6..)  . 
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The  matrix  P 


(pu> 


will  be  written  as 


A 

T 


(3.4) 


where  P  =  (p. .)  corresponds  to  states  for  which  i,  j  €  T  and  Q  ■•=  (p„) 
correspond  to  states  for  which  i  «  T  and  j  «  A  . 

The  first  problem  of  interest  for  our  model  is  the  calculation  of  U(t)  . 

We  shall  first  show  that  U(t)  is  related  to  w(t),  and  then  show  that  w(t) 
is  the  solution  to  a  matrix  renewal  equation.  The  components  of  U(t)  satisfy 
the  equation  * 


fc^t  -  t)  dr 


(3.5) 


or,  in  matrix  notation 

0  r 

U(t)  =  XJ  (t)  +  J  u(t)  P  $(t  -  T)dT  .  (3.6) 

0 

The  derivation  of  (3.5)  is  immediate.  If  the  patient  is  in  state  i  at  time  t  , 
he  is  either  in  that  state  for  the  first  time,  or  the  last  transition  took  place  at 
some  time  t  to  an  occurrence  of  state  i  which  is  still  in  progress.  It  can  be 
observed  from  (3.6)  that  U(t)  is  simply  related  to  w(t)  and  other  known 
functions.  The  matrix  «(t),  in  turn,  can  be  formed  as  the  solution  to  a  matrix 
renewal  equation,  which  reads,  in  component  form 
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(t)  =  “?(t)  +  Tj  I  WW(T)  Pui  "  T)dT  (3.7) 

1  1  k«T  0  K  141  1 

where  w^(t)  can  alternately  be  expressed  as 

“0(t)  =  U^CO}  ?i°(t)  (3.8) 

and  the  (t)  are  known  initial  values  of  the  probability  of  being  in  state 
i  .  The  equivalent  matrix  representation  of  (3.  7)  is 

0  } 

w(t)  =  w  (t)  +  J  w(t)  P  <p[t  -  T)dT  .  (3.9) 

0 

The  derivation  of  (3.9)  is  similar  to  that  of  (3.  5)  . 

Since  the  fundamental  equations  for  the  description  of  our  model  are  in 
convolution  form,  the  use  of  Laplace  transforms  casts  them  in  algebraic  form 
and  facilitates  the  calculation  of  moments.  The  Laplace  transform  of  any 
function  of  time  will  be  denoted  with  an  asterisk  and  an  argument  s  e.g. 

00 

w*(s)  =  /  e  Stw(t)dt  (3.10) 

0 

where  the  Laplace  transform  of  a  matrix  is  the  same  matrix  containing  as  elements 
the  transform  of  the  original  elements.  With  this  convention  (3.6)  and  (3.9) 
yield  the  results: 

#  0*  *  -  * 

u  (s)  =  u  (s)  +  M  (s)  P<p  (s)  (3.11) 

*  0*  *  * 

U  (s)  =  U  (s)  +  w  (s)  P  $  (s)  . 


(3.12) 
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^  t 

The  first  line  can  be  solved  for  u  (s)  when  Re(s)>0  since  |I-P^>  (s)|*0 

*  - 

in  the  right  hand  s  plane.  To  see  this,  we  note  first  that  I  -  ?<p  (0)  =(6^  -  p^) 

4 

We  further  have  the  condition 


1  i  ?  hi 


(3.13) 


-  * 

where  strict  inequality  holds  for  at  least  one  value  of  i  .  Hence  I-P<p  (0) 
has  a  weakly  dominant  main  diagonal  and  its  determinant  therefore  differs  from 
zero,  [6].  When  Re(s)  >  0  we  know  that  |y^(s)l  <_1  ,  hence  the  inequality 
of  (3.13)  is  strengthened. 

The  solution  to  the  relations  of  (3.11)  and  (3.12)  can  therefore  be  written 


“*  (s)  =  [I  -  p/ls)]"1 

U*(s)  =  U°*(s)  +  o°%)  [I-p/cslf1  P  ts)  .  (3.14) 


It  can  easily  be  shown,  following  Feller's  proof  for  the  ordinary  renewal  equation 
[7],  that  the  elements  in  «(t)  which  are  derived  by  inverting  the  Laplace 

3jc 

transform  w  (s)  are  all  non-negative.  If  the  set  A  is  void,  i.  e.  ,  there  are 
no  absorbing  states,  then  we  can  use  standard  Tauberian  techniques  to  derive 
asymptotic  expressions  for  w(t)  and  U(t)  . 


4  In  the  set  A  is  void,  similar  considerations  lead  to  the  same  conclusion  for 
for  Re(s)  >  0  . 
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4.  First  Passage  Times 

We  now  turn  to  first  passage  time  problems  which  are  of  great  importance 
in  applications  involving,  for  example,  the  comparative  efficiency  of  different 
drugs  in  ameliorating  the  course  of  a  chronic  disease.  In  the  particular  problem 
which  we  studied,  the  treatment  of  acute  leukemia,  a  fatal  end  is  inevitable 
and  the  set  A  consists  of  a  single  state.  It  is  then  plausible  to  base  a  test 
of  efficiency  of  a  drug  on  the  amount  of  time  that  a  patient  is  kept  alive. 

Another  alternative  would  be  to  base  efficiency  on  the  amount  of  time  that  a 
patient  is  kept  in  a  state  of  relative  comfort. 

The  first  question  that  might  reasonably  be  asked  is,  how  long  will  the 

* 

patient  remain  in  one  of  the  disease  states?  Let  f(t)  be  the  probability  density 
for  the  time  to  reach  some  absorbing  state.  Then  we  have 

f(t)  =  2  2  «i(t)  P4i,=  «(t}  Q  (4.1) 

it  T  jc  A 

Questions  relating  to  the  particular  absorbing  state  in  which  the  patient 
finally  finds  himself  can  be  answered  by  reference  to  well-known  results  from 
the  theory  of  finite  Markov  chains  since  these  results  are  independent  of 
duration  of  stay.  For  this  purpose,  let  H  be  a  single  absorbing  state  and  Q 

H 

be  the  column  vector  in  A  corresponding  to  state  H  .  Then  the  probability 
that  a  patient  will  eventually  end  in  the  absorbing  state  H  is 

qR  =  U°(I  -  P}'1  Qh  .  (4.2) 
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When  A  consists  only  of  the  single  state  H,  it  is  easily  verified  that 

5 

q  =  1  .  The  probability  density  function  for  the  amount  of  time  spent  in  the 

H 

transient  states  T  conditional  on  final  absorption  by  H  is 

w(t)  Qh 

V»— ^  (4.3) 

rl 

We  now  proceed  to  calculate  expressions  for  the  moments  by  using  the 
* 

value  of  «  (s)  given  in  (3.14)  as  a  moment  generating  function.  The  kth 
moment  for  the  total  stay  in  the  set  of  T  states  conditional  on  absorption  by  H 
is 

E(tk)  =  A]c  QR/qH  (4.4) 

where  the  A^  are  defined  recursively  by 

Ak+1  (I-P)  =  U°  M°  (k  +  l}+(kJ'1)  A]cPM(l) 

+  (k21)  A^_2  P  M(2)  +  ...  +  U°  (I  -  P}-1  M(k  +  1)  .  (4.5) 

5  We  are  indebted  to  Dr.  Joan  Rosenblatt  for  pointing  out  that  if  A  is  a  single 
state  than  Q  can  be  written  in  alternative  form  as 

Q  =  (I  -  P) 

00 

From  this  form  it  is  easy  to  see  that  /f(t)dt  =  l  ,  since  w*(0)  =  (I-P) 

0 
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The  first  two  of  the  A^  are  explicitly 
Aj  =  U°  {M°(l}  +  Cl  -  Pf 1  P  M(l}>  (I  -  Pf 1 
A2  =  U0{M°(2)  +  (I  -  P)"1  P  M(2)  + 

+  2[M°(1)  +  (I  -  P}_1  P  M(1}](I  -  P)-1  P  M(l)>  (I-P)"1  (4.6) 

Formal  expressions  for  the  higher  values  of  the  subscript  are  easily  derived 
in  this  fashion.  When  the  set  A  consists  only  of  H,  there  is  some 
simplification  afforded  by  using  the  expression  for  Q  in  terms  of  (I-P)  , 
since  this  term  will  cancel  (I-P)  1  which  post-multiplies  the  expression  for 


5.  Distribution  of  Time  Spent  in  a  Single  State 

5.1  The  p.  d.  f.  <p^.  (t)  =  (t).  One  further  distribution  that  has  potential 

application  in  drug  comparisons  is  the  distribution  of  time  spent  in  a  single  one 
of  the  transient  states.  We  shall  first  find  the  distribution  of  the  time  spent  in 
the  particular  transient  state  i(i  e  T),  on  the  assumption  that  ^(t)  =  ^(t)  . 
Let  the  partition  of  the  transition  matrix  P  be 


P  = 


A 

I 

6 

Y 


i 

O 

O 

a 


T-j 

O 

P 

n 


A 

i 

T-i 


(5.1) 
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Thus  a  is  a  vector  which  contains  all  the  information  about  transitions  from 
T  -  i  to  i;  p  is  a  vector  of  transition  probabilities  from  i  to  T  -  i;  etc. 

Note  that  the  probability  of  a  state  beginning  in  T  -  i  will  lead  to  state  i 
before  it  leads  to  an  absorbing  state  is 

2  -1 

R  =  a+  n  n  <*+...  =  (I  -  n)  a  ,  (5.  2) 

^et  ^ii  be  the  probability  that  a  patient  once  in  state  i  will  return  to 
state  i  before  reaching  an  absorbing  state.  Further,  let  h  be  the  probability 
that  a  patient  in  state  j  will  reach  state  i  before  reaching  an  absorbing  state. 
We  shall  find  the  Laplace  transform  for  the  time  spent  in  state  i  ,  starting 
from  state  j  , 

^(s)  =  2  P,(n)  v*(s)  (5.3) 

J  n=0  ■ 

where  Pj  (n)  is  the  probability  that  the  patient  is  in  state  i  exactly  n  times 

* 

beginning  from  an  occurrence  of  state  j,  and  vn(s)  is  the  Laplace  transform  of 
the  probability  density  for  the  total  time  spent  in  occurrences  of  state  i  . 

The  parameters  h^  and  h.^  are  given  in  the  notation  of  (5.1)  by 

h  =  pn  (i  -  n)"1  a=  pn  r 

hJi  =  (R)j=[(i-n)"1o-]j  (5.4) 

The  derivation  of  h  follows  immediately  from  the  definition  of  R;  the 
derivation  of  h  follows  from  the  expansion 
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hii  "  ^  j^T_i  ^Jk  \  +  Pj  ^  *jk  \  +  *  *  *  * 


(5.5) 


The  patient  starts  in  state  i,  makes  a  transition  to  state  j  with  probability 


p, ;  remains  in  T  -  i  one  or  more  epochs  ending  in  state  k  with  probability 
[n  (I  -  II)  *]  ,  ana  finally  returns  to  i  with  probability  . 

The  probability  that  there  are  exactly  n  occurrences  of  state  i  is 

r 


hn_1  (1  -  h. .), 
n  v  n'  ’ 


Pj(n)  =  (  1-h..,  j  #  i, 


J  =  i 

n  =  0 


h..  hn.-1(l  -  h..),  j*i,  n>  1  .  (5.6) 

ji  li  ir’  ’  ~ 


Hence,  we  find  for  (s)  the  expression 


0* 

*  C1_hiiK  (s)  6u  i  -u*- 

».(«)= — ‘M; - +  — SH — >(*-'«)•  l5-7) 

J  i  _  u  r~\  1  J1  i  u  .  \  xj 


h.{  (1  _  h  .)  <p  (s) 


1  -  h.,  <p.  (s) 


1  -h. .  <p.  (s) 
ii  i 


If  we  denote  the  kth  moment  of  the  total  sojourn  time  in  state  i  by  a  patient 


who  starts  out  in  state  j  by  ^(k)  we  find  for  the  first  two  moments 

0  hiimiC1>  h  (1-6  )m  (l) 

H..  Cl)  =  [m  (I)+  ■  I  5..+  - 1 - 

rji  i  1  -  h.,  J  lj  1-h.. 

ii  ii 


2m  (1)  m  (1)  h  +  h  m  (1) 

=  - i7h  U  + 


ii 


2^  tmttl)]Z 


(‘  -  hU> 


m,(2)  [m,(l)] 

r^l6i)+vi-v[Ar, *  •  l5-e> 


ii 


d-hii)‘ 
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5.  2  General  Case  .  The  more  general  case  for  determining  the  distribution 
of  total  time  spent  in  the  transient  state  i  when  the  sojourn  time  distribution 
<p  (t)  depends  on  the  state  immediately  following  is  quite  complicated.  In 
this  section  we  will  only  indicate  the  nature  of  the  results.  Let  ^(t)dt  denote 
the  probability  density  for  the  event  that  the  patient  is  in  state  i  (i  e  T)  for 
an  amount  of  time  in  (t,  t+dt)  .  Also  define  W  to  be  the  vector  of 
probabilities  that  a  chain  of  states  starting  from  some  state  j  e  T  -  i  will  end 
in  absorption  without  leading  to  state  i  .  The  expression  for  W  is  seen  to 
be  * 

w  =  y  +  nv  +  n  Y  +  (5. 9) 


on  the  assumption  that  A  consists  of  a  single  state. 

Then  the  Laplace  transform  of  £  (t)  can  be  written  as  the  sum  of  two 
components 


(5.10) 


where 


.0*  * 

*  0  Jo*  0*  [x  Cs)R][K  (s)  W] 

lijW  =  U;W[\0  (s)  WJA  +  6\^(s)  + - 1 - A 


l-[\  {s)R]i 


(  Z  U°(0)R  )  [X* C s)  W] 

4  J  J  i 


1  -  [V  (s)  R] 


(5.11) 
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and  where 


*  *  n*  n* 

^  (S)  =  (Pij  ^j  (s)),  X.  (s)  =  (p. j  <PLi  (s)),  i,  j  €  T 


0*  0*  *  v->  0* 

Sa(s)  =(pij  ^iA^’  *iA(s>  =  .^A  Pij  %  CS)  * 


(5.11a) 


The  moments  of  the  duration  of  time  spent  in  state  i  are  obtained  by 

* 

successive  differentiation  of  £^(s)  .  Defining  the  vectors 


nk=(pijmU(1:»’  nk=(p«mU(k» 


15.12) 


the  first  two  moments  can  be  written 


of  o  (n  R)(pw>  +  (pR)(n  w) 

^.)  =  U.  ^  W  +  6  m1A(l)  +  ~  i~I~pR - ~ 


(PR)CPW)(n.  R) 


(i-PR) 


!1R)  „  0  raw  (pW)(BR) 

J  +t^W?V[l=?R  * 


Cl  -  PR) 


r  (n°  r)  (pw>  +  2  (n?  r}  (a  w)  +  CPR)  (a,  w) 

E(tf)  =  U°  W  +  6m..(2)  +  — i 1 2 - 

v  x '  l  2  iAv  '  1  -  PR 


+ - 3 - 2  R)(PW)(a  R)  +  2(j3R)(a  R)(a  W)  +  (PR)(PW)(«  R)] 

(1  -PR)  11  11  L 


• - (PRXPWHaR)2  +(Zu%.} 

(1-PR)3  1  .  j*i  >  J  1  PR 


2  (fi1  W)(£21  R)  +  (P W)  (O  R)  2  (p W)  (O  R) ‘ 

+  - - -  +  - r— 

(l  -  PR)  Cl  -  PR) 


(5.13) 
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6.  Application  of  the  Model  to  Patients  With  Acute  Leukemia 

6.1  Outline  of  the  six  state  model.  A  large  scale  set  of  clinical  trials 
was  conducted  by  Acute  Leukemia  Group  B  (Frei  et  al.  [8])  on  patients  having 
acute-leukemia.  In  this  section  we  will  apply  our  semi-Markov  model  to  a 
portion  of  the  data  from  these  clinical  trials. 

The  data  considered  here  are  from  the  records  of  54  patients  with  lymphocytic 
leukemia.  All  patients  entered  these  clinical  trials  while  in  a  state  of  relapse. 

The  patients  received  therapy  and  either  continued  in  relapse,  expired,  or  had 
a  partial  or  complete  remission.  The  stay  in  a  remissive  state  is  only  temporary 
and  the  patients  ultimately  suffer  a  relapse.  Additional  therapy  is  given  and 
the  patients  either  expire  or  again  reach  a  remissive  state.  Ultimately  all 
patients  expire  as  the  effect  of  the  therapy  is  only  transitory  .  Among  the 
patients  in  the  study  discussed  here,  no  more  than  two  remissions  were  attained 
by  any  patient.  The  therapy  is  described  by  Frei  et  al,  [8],  and  essentially 
consists  of  two  phases.  Phase  I  consists  of  initially  giving  the  patients 
methotrexate  (MTX).  Phase  II  consists  of  therapy  with  6-mercaptopurine  (6-MP). 
Phase  I  was  continued  for  all  patients  in  the  initial  relapse  state.  If  at  the  end 
of  six  weeks,  the  patients  had  not  responded  to  the  treatment  (and  were  still 
alive),  the  treatment  was  terminated.  A  lapse  of  two  weeks  was  allowed  after 
which  the  patients  were  placed  in  Phase  II.  The  particular  phase  which  the 
patient  was  receiving  was  discontinued  when  the  patient  relapsed  from  a  remission 
state.  The  patient  was  then  placed  in  the  alternate  phase. 
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In  our  application  we  will  be  mainly  interested  in  the  distribution  of 
the  time  to  failure.  Further,  since  a  remissive  state  is  always  followed  by  a 
relapse,  we  will  combine  the  sojourn  time  in  a  relapse  state  with  that  in  the 
remissive  state  which  immediately  preceded  it.  Also  the  data  seem  to  indicate 
different  characteristics  for  the  sojourn  times  in  the  various  states.  These 
are  a  function  of  the  number  of  times  the  patient  has  been  in  the  state. 

With  these  characteristics  of  the  process  in  mind,  we  define  the  six  states: 

SQ:  failure  (death) 

Sji  initial  relapse  state  (condition  of  patient  on  entering  study) 

S2:  first  partial  remission  (also  includes  subsequent  relapse) 

S^:  second  partial  remission  (includes  subsequent  relapse) 

S^:  first  complete  remission  (includes  subsequent  relapse) 

S5:  second  complete  remission  (includes  subsequent  relapse) 

The  communication  between  states  is  summarized  in  Figure  1. 

Some  typical  case  histories  are: 


Patient  number  Condition  Duration  of  Stay  in  State  ( weeks) 

14  initial  relapse  12 

failure 

88  initial  relapse  2 

first  partial  remission  34 

relapse  10 

failure 

284  initial  relapse  4 

first  complete  remission  13 

relapse  24 

second  partial  remission  40 

relapse  8 

failure 
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v  Applying  equation  (3. 14}  with  five  transient  states  results  (after  some 

*  * 

algebra)  in  the  Laplace  transforms  for  w  (s)  and  U  (s);i.  e. 

w*(s}=<p*(s) 

w^sJ  =  p12*i*(s)*2(s) 

j|(  j||  j||  a|i  jfc  % 

“3(s)  =  P12  P23^ltSJ  +  P14P43’’l^S^  ^3^ 

]j|  )||  iff 

w4(s)  =  P14  (s)  <e4(s) 

j|l  j||  jj(  ^  ^  )||  j)( 

w5(s}  =  P12  P25  <PX  (s)  *>2(s)  ^5CS)  +  P14P45  (s)  *5^ 


>  (6.1) 


U*(s)  =  ®*(s) 

U2CS}  =  P12  ®2(si 

U*CS)  =  P12  P23^>*(s)  f2(s)  ®*(S}  +  P14  P43  **( s}  *>4(s)  ®*(s) 
uj.)  =  P14  <PX  (s)  **(S) 

9^C  ^  ^ 

U5(s)  =  P12  P Z5<P1  (S)  <P2{ s)  «5(s)  +  P14  P45  ^  (s)  ^4(s)  *5C.l 


The  probability  density  function  fQ(t)  of  the  time  to  reach  SQ  from 
can  now  be  obtained  from  (4.  3)  with  QH  =  (p1Q,  P2Q,  P3Q,  P4Q,  p5Q}  . 


Hence  we  have 
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5 

f0W  *  Z  wiW  pio  C6‘2) 

i=l 

where  w^(t}  can  be  obtained  by  taking  the  inverse  Laplace  transform  of 
w*(s)  . 

The  moments  for  the  time  to  reach  the  failure  state  can  now  be  immediately 
written  down  by  using  equation  (4.4)  ECt?4)  =  Qq  with  the  recursive  relation 
(4.  5)  .  The  results  for  the  mean  and  variance  are 

E(t)  =  m^l)  +  p12  m2(l)  +  m3(l)  [p12  p23  +  pJ4  p43]  +  p^  m4 
+  m5  fp!2  P25  +  P14  P45* 

a  2  W  =  cr  f  +  P12  cr  2  +  a  <r  2  +  p14  «r  2  +  b  ^  +  p12  (1  -  p12)  m2  (1) 

+  a(l-a)  m2(l)  +  P14(l  “  P14)  m2(l)  +  b(l-b)  m2(l) 

+  2  |m2 {1}  m3(l)  [p12  (p23  -  a)]  -  m2(l)  m4(l)  pJ2  Pj4 
+  m2(l)  m5(l)  [p12  (P25  -  b)J  +  m3(l)  m4(l}  [p14(p43  -  a)] 

-  m3(l}  m5(l)  ab  +  m4(l)  m5(l)  [p]4  (p45  -  b)J 

2  2 
where  <r  i  =  m  (2)  -  m^l)  ,  and 

9  =  P12  P23  +  P14  P43’  b  =  P12  P25  +  P14  P45  ’ 


(6.3) 
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In  applying  the  data  to  the  model  it  is  of  interest  to  study  the  distribution 
to  reach  the  failure  state,  conditional  on  the  patient  also  reaching  a  remission 
state.  Denoting  the  p.  d.f.  of  this  conditional  distribution  by  gQ(t),  we  have 


«o“>  * 


tpM  -  ^(t)  p10 
1  "  P, 


10 


=  2  ^(t)  p10/(i  -  p10) 


(6.4) 


The  mean  and  variance  of  the  time  to  reach  the  failure  state  (6.  3)  is 
similarly  modified  by  replacing  piQ  by  piQ/(l  “  P1Q}  • 

Another  complication  in  applying  the  model  is  that  the  treatment  given  the 
patients  depended  on  the  length  of  time  they  remained  in  the  initial  relapse 
state.  Those  patients  who  stayed  in  the  initial  relapse  state  for  a  period  of 
eight  weeks  or  less  were  in  Phase  I,  whereas  those  who  remained  for  more  than 
six  weeks  received  Phase  I  therapy  followed  by  Phase  II  therapy. 

6. 2  Estimation  of  parameters  .  It  remains  to  estimate  the  parameters  in 
the  model.  These  are  the  (p^)  and  the  sojourn  time  distributibn  for  the  various 
states.  The  relative  frequencies  observed  in  the  clinical  trials  were  used  as 
estimates  of  the  p^  .  The  numerical  results  are: 
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The  subscripts  I  and  II  refer  to  those  patients  that  either  were  in  Phase  I  or 
II  when  they  left  the  initial  relapse  state. 

An  investigation  of  the  distribution  of  sojourn  times  within  the  various 
transient  states  showed  that  these  distributions  could  very  well  be  approximated 
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by  gamma  distributions;  i.  e. 

...  UUlf  *  -\.t  . 

^>(t)  =  ®  ■  »  a >  0  .  (6.5) 

The  parameters  (a,X)  were  estimated  by  the  method  of  maximum  likelihood  with 
the  aid  of  the  very  convenient  tables  of  Wilk,  Gnanadesikan,  and  Huyett,  [9]  . 
Due  to  the  small  number  of  patients  in  group  II,  we  decided  to  combine  the 
sojourn  time  data  for  groups  I  and  II.  Table  I  summarizes  the  results  of  these 
calculations. 


TABLE  I:  Maximum  Likelihood  Estimates  of  the  Parameters  of 

the  Gamma  Distribution 


State 

a 

V 

n  (sample  size) 

S2  (first  P.R.) 

2.  33 

0.101 

14 

S3  (second  P.R.) 

2.87 

.122 

7 

S4  (first  C.R.) 

4.71 

.132 

27 

S5 (second  C.R.) 

15.54 

.463 

7 

• 
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Table  II  summarizes  the  sample  means  and  variances  for  the  two  groups. 


TABLE  II:  Sample  Means  and  Variances  (Patients  reaching  remission  only) 


m(l) 

.2 

<r 

n  (sample 
size) 

I 

II 

i 

II 

I 

II 

Sj  (initial  relapse) 

4.2 

12.6 

1.7 

11.7 

26 

15 

S2  (first  P.  R. ) 

19.2 

33.  0 

124 

301 

10 

4 

S3  (second  P.  R. ) 

33.0 

— 

215 

— 

7 

0 

(first  C.  R. ) 

38.6 

31.4 

435 

260 

16 

11 

S5  (second  C.  R. ) 

34.8 

31.0 

98 

128 

5 

2 

The  initial  relapse  state  sojourn  time  distribution  (conditional  on  reaching 
remission)  for  groups  I  and  II  are  both  truncated  distributions.  The  distribution 
for  group  I  is  truncated  at  the  upper  tail  at  eight  weeks;  whereas  group  II  is 
truncated  at  the  left  tail  at  eight  weeks.  Since  the  observations  are  in  discrete 
time  units  (weeks)  this  truncation  problem  can  be  handled  by  taking  these  sojourn 
time  distributions  to  be  discrete  distributions.  The  estimates  of  the  point 
probabilities  used  were  the  observed  relative  frequencies.  These  relative 
frequencies  are: 


Weeks  I  (II) 

1(9) 

2(10) 

3(11) 

4(12) 

5(13) 

6(14) 

7(15) 

(18) 

(22) 

I 

P  = 

0 

2/26 

6/26 

8/26 

5/26 

4/26 

1/26 

0 

0 

II 

P  = 

2/15 

2/15 

1/15 

5/15 

2/15 

1/15 

0 

1/15 

1/15 
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The  weeks  in  parentheses  refer  to  the  times  for  group  II. 

A  check  on  our  model  can  be  obtained  by  comparing  the  sample  mean  and 
variance  of  the  time  to  reach  failure  with  the  theoretical  formulae  given  in  (6.  3). 
The  numerical  results  (conditional  on  going  to  remission)  are: 


I 

Data  Model 

II 

Data  Model 

Mean 

48.4 

49.7 

48.  5 

48.6 

Variance 

515 

525 

358 

408 

The  agreement  is  very  good. 

We  now  turn  our  attention  to  estimating  the  distribution  of  the  time  to  reach 
the  failure  state  (conditional  on  reaching  a  remissive  state).  The  probability 
density  function  can  be  written  explicitly  from  (6. 2)  and  (6. 4);  i.  e. 

90(t)  =(1-P10)  +  P30P12P23^1*^2 

+  p40  P14  [*1  *  *4J  +  P30  P14  P43  [*1**3*  U] 

+  P50  P12  P25  ^1  *  *2  *  *5]+  P50  P14  P45  ^1  *  H  *  *5]}  (6*  6) 

(The  notation  <p^  <p2  denotes  the  convolution  of  ^(t)  and  <p2{ t)  etc.).  Note 
that  gQ(t)  is  made  up  of  a  mixture  of  distributions  which  involve  convolutions 
of  gamma  distributions.  We  approximated  the  convolution  of  gamma  distributions 
by  a  gamma  distribution  having  the  same  first  two  moments.  It  was  felt  that  a 
more  sophisticated  approximation  was  not  necessary. 
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Figure  II  compares  the  cumulative  distributions  obtained  from  the  model 
with  the  empirical  cumulative  distribution  for  both  groups  I  and  II.  Both  the 
empirical  and  theoretical  distributions  for  II  have  been  adjusted  by  using  8 
weeks  as  the  origin.  The  agreement  of  the  model  with  the  data  is  excellent 
for  group  I.  However  the  agreement  at  the  lower  tail  for  group  II  is  not  good. 
This  probably  is  due  to  the  nature  of  the  data  where  at  the  early  stages  of 
phase  II  treatment,  there  are  undoubtedly  some  effects  from  phase  I  still 
present.  Also  on  Figure  II,  the  failure  time  distribution  for  patients  not 
going  into  remission  is  given  for  comparison. 
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